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brief. 
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IN THE UNITED S TA TES PATENT AND TRADEMARK OFFICE 


Patent Application Scr. No.: 09/822,121 


Group Art Unit: 2643 


Filing Date: 3/30/2001 


Examiner: WTNG l\ C1LAJM 


Attorney Docket Number PII-US 010080 Inventor Namc(s): COLMENAREZ FT AL. 
Confirmation #: 8881 

Title: METHOD AND APPARATUS FOR AUDIO/IMAGE SPEAKER DETECTION AND 
LOCATOR 

Mail Stop Appeal Brief 
Commissioner for Patents 
P.O.Box 1450 
Alexandria VA 22313-1450 


Tliis is an appeal from the final rejection of Claims 1 -25. 

I REAL PARTY IN INTEREST 

The real party in interest is Koninkiijkc Philips Electronics, N.V., a corporation of the 
Netherlands. 

II. RELATED APPEALS AND INTERFERENCES 

Applicants arc not aware of any related appeals or interferences. 
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IU. STATUS OF CLAIMS 

Claims I -8 and 1 0-24 stand rejected under 35 USC 103(a) over WO 99/60788 ("Potts") 
in view of US 6,704,048 ("Malkin") - section 8 of the final office action. 

Claims 9 and 25 stand rejected under 35 USC 103(a) per the other claims and further in 
view of US 5,778,082 ("Chu") - section 9 of the final office action. 

There arc no other claims. 

All rejected claims arc being appealed. 

fV. STATUS OF AMENDMENTS 

There was a first communication under rule 1 16. In a first advisory action, the Examiner 
said that this first communication overcame all rejections over Baker. In a second advisory 
action, correcting the first advisory action, the Examiner said that the firsL communication under 
rule 1 16 could not be entered due to an error in the claim identifier for one of the claims. 

In a telephone conversation dated October 19, the Examiner indicated that if Applicants 
corrected the claim identifier, all rejections other than paragraphs 8 and <) of die office action 
would be overcome. 

A second communication under rule 1 16 was faxed in on October 20, 2004, correcting 
the problem with the claim identifier, repeating the amendments to the claims, and incorporating 
the prior argument, by reference. Applicants therefore believe that the only parts of the office 
action that remain to be overcome are paragraphs 8 and 9, though Applicant* have noi yot 
received a third advisory action staling this in writing. 

Accordingly, in reliance on the Examiner's statements during die telephone interview, 
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Applicants will only argue againsl sections 8 & 9 of the office action. 

V. SUMMARY Ol? TJTR CLAIM KD SURJKCT MAT112R. 
Claim l 

The claimed invention relates to a video conferencing system. The system includes a 
stationary image pickup dev.ee (rcf #2 1 0; spec p. 4, 11. 1 2, 1 8, 20; p. 5, 11. 3-1 0 & 19; p. 6, 1. 1 0). 
The image pickup device remains motionless during operation (spec, p- 5, II. 4, 8, 9, 19). 1 he 
system also includes an audio pickup device (rcf. #\s 231, 232; spec. p. 4, 1. 13-14; p. 5, 1. 19) lor 
generating audio signals (ref. # 235; spec. p. 5, 1. 20) representative of sound from an audio 
source. The system also includes means (rcf. # 270; spee. p. 5, L 20 through p. 6, 1. 9) for 
processing the image signals and audio signals to determine a direction of the audio source 
relative to a reference point. The determination of direction depends al least at times on the 
image signals. 

Claim 5 

This claim recites the video conferencing system of claim 1, further comprising an 
electronic pan tilt zoom system for electronically manipulating the image signals to effectively 
provide at least one of variable part, tiJt, and zoom functions, (spec. p. 6, 11. 10-1 5) 

C laim 2 

This claim recites the video conferencing system of claim 1 , wherein said proeessing 
means (ref. #270; spee. p. 5, 1. 20 through p. 6, 1. 9) comprises: 
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an audio source localization system (ref. #240, spec. p. 5, 1. 21); 

a computer vision person detection system (ref. #250, spec. p. 5, 1. 22); and 

a multimodal speaker detection system (ref. #260, spec. p. 5, 1- 22 through p. 6, 1. 9). 

Claim 3 

This claim recites the video conferencing system of claim 2, further comprising an 
integrated homing (ref. #110, spec. p. 6, 1. 14 ct. «cq.) for an integrated video conferencing 
system incorporating the image pickup device (ref #210; spec p. 4, 11. 12, 18, 20; p. 5, 11. 3-10 & 
19; p. 6, 1. 10), the audio pickup device (ref. #s 23 1, 232; spec. p. 4, 1. 13-14; p. 5, 1. 19), and the 
processing means (ref. #270; spec. p. 5, 1. 20 through p. 6, 1- 9). 

(Maim 4 

This claim recites the video conferencing system of claim 3, wherein the integrated 
housing (ref. #1 1 0, spec. p. 6, 1. 14 et. scq.) is sized for being portable. The portability is a 
functional advantage. 

Claim JS 

This claim recites the video conferencing system or claim 1 , wherein the image pickup 
dev,ce (ref #210; spec p. 4, 11. 12, 18, 20; p. 5, 11. 3-10 & 19; p. 6, 1. 1 0) is a stationary camera 
that remains motionless during operation of the video conferencing system. 
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Clai m K 

This claim recites lhc video conferencing system of claim 2, wherein the audio source 
location system (ref. #240, spec. p. 5, 1. 21) detects the movement of the audio source when 
the audio source moves relative to the reference point, and, in respotise to the movement, lhc 
audio source localization system causes a change in a Held of view of the image pickup device 
(spec. p. 5, 11. 8-9)- It should be noted that this claim still depends from claim 1, which stales 
that the image pickup device is motionless. 

Claim 10 

This claim has recitations analogous to those of claim 1 and further recites manipulating 
the image signals to produce refined image signals (spec. p. 3, 11. 4-9) depending on the 
determined direction; and outpuiting this refined image sienals. lixamplcs of refined image 
signals are discussed in the specification further with respect to simulating pan/tiltfcoom 
functions as discussed with respect to claim 5. 


Cl aim 1 1 

This claim depends from claim 1 0 and contains recitations analogous to those of claims 2 

and 5. 
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Claim 12 

'I 'his claim depends from claim 1 0 and recites that manipulating the image signals 
includes varying a field of view of the image pickup device in response to the control signals. 
The spec uses the pan/tilt/zoom feature as an example of varying the Held of view, as explained 
with respect to claim 5. 

Claim 1 4 

This claim depends from claim 10, but contains recitations similar to those of claim 8. 
The reader is referred to the summary of claim 8, above. 

Claim IS 

This claim recites the method of claim J 3, wherein processing the imago signals includes 
generating control signals (265, spec. p. 6, 11. 1-2) depending on the audio based direction, and 
manipulating the image includes electronically panning, tilting, and/or zooming said image 
pickup device depending on the control signals (spec. p. 6, 1L 10-15). 

Claim 16 

This claim recites a video conferencing system comprising: 

microphones lor generating audio signals representative of sound from a 

speaker; 

a stationary video camera, remaining motionless during operation, for 
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gcncruLing video signals representative of a video image; 

an electronic pan tilt zoom system for manipulating video images 10 
produce the visual effects ol'pannirg, tilting, and/or zooming; 

a processor for processing the video signals and the audio signals to 
determine a direction of a speaker relative to a reference point and supplying 
control signals to the electronic pan tilt zoom system for producing images that 
include the speaker in ihc field of view of the camera, the determination of 
direction depending at least at times on the video signals, the control signals being 
generated based on the determined direction of the speaker; and 

a transmitter lor transmitting audio and video signals lor video 
conferencing. 

Accordingly, thi- claim includes, /»/«- alia, the siationaiy/motionlcss limitation discussed 
with respect to claim I and the pan/tilt/zoom system limitation discussed with respect to claim 


5 l . 


Cla im 1 7 

This claim recites the video conferencing system of claim 1, wherein at times the 
determination of the direction of the audio source depends on both the image signals and the 
audio signals (spec. p. 5, line 1 8 through p. 6, line 9). 


' The cl*m is different in scope from claim 5, however, because it docs not contain the h^on 
frZ cMm t^^tcmitaatkm of direction depends at least at times on the tmage signal s. 
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Cl aim 18 

This claim recites the video conferencing system of claim 1, wherein the processing 
includes determining the movement of the audio source depending at least at Limes on the image 
signals, (spec. p. 3, 1. 7; p. 5, 1. 18 through p. 6, 1. 9). 

Claim 19 

This claim recites the video conferencing system of chum I, wherein the processing 
includes tracking the position of the audio source when the audio source moves, the tracking 
depending at least at times on thcimage signals (spec-, p. 3, 1. 7; p. 5,1.18 through p. 6, 1. 9). 

Claim 20 

This claim recites the video conferencing system of claim 2, wherein the computer vision 
person detection system detects the movement of the audio source when the audio source moves 
relative to the reference point, and, in response to the movement, the computer vision person 
detection system causes a change in a field of view of the image pickup device, (spec. p. 3, 1. 7; 
p. 5, 1. 18 through p. 6, 1. 9) 

Claim 21 

This claim recites the method of claim 1 0, wherein processing the image signals farther 
includes: 

detecting the movement of the audio source when the audio source moves; 

and 
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causing electronically, in response to the movement, an Fsie] variation in a field of 
V1 ew of the image pickup device, (spec. p. 3, 1. 7; p. 5, 1. 18 through p. 6, 1. 9) 


Claim 2 2 

The recitations of this claim arc similar to tho.se of claim 18, except that, the claim 
depends from claim 10, rather than claim 1 . 

Clai m 23 

The recitations of this claim are similar to those of claim 1 9, except thaL the claim 
depends from claim J 0, rather than claim 1 . 


Cl aim 2 4 

This claim recites a video conferencing system, but is otherwise analogous to claim 10. 


VI. GROUNDS OF REJECTION TO BE REVIEWED OK APPEAL 

Section 8 of the final office action is to be reviewed, per the telephone conference of 
October 19. wherein the Examiner indicated that the prior sections would be overcome by the 

amendment of October 20. 

Section 9 of the office action would appear to be moot. 
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Vn. THE ARGUMENT 

Section K of the Office Action purports to reject 23 claims over a combination of 
references. The references are long and complex. The Potts document contains 70 pages and 
25 sheets or figures. The Malkin document contains 12 columns of text and 8 sheets of 
drawing. 2 Therefore, in keeping with 37 CP.R. 1 .104 (c) <ii) tbe Hxaminer is required to specify 
what part of the references is relied upon in rejecting each claim. 

Instead, the Examiner has grouped all of the rejections together, without indicating which 
element of which claim is rejected over which part ol the references. The comments made arc 
not clearly applicable to all claims. Accordingly, Applicants respectfully submit that the 
rejections arc improper. 

The Potts reference: claim 1 

Applicants respectfully submit that paragraph 8 of the Hnal office action fails to make a 

prima facie case of obviousness against claim 1 . 

The Examiner mischaractcrizc* Applicants' amendment of February. 2004, by stating that 
page 7. second & third paragraphs, makes admissions as to the Potts reference. This is fidsc 
Those paragraphs of the amendment do not refer to the Potts reference at all. Those paragraphs 
ofthe amendment refer to the summary section of Applicants' own specification at pages 2 and 
3, and therefore do not characterize any prior art. 


> The sveion -l*o ciu* the Baker however, .inc. the Uxaxniner has indicated that the lialccr reference is 

withdraw, Applicants -ill only discus, the Potts j^n«jTO. j „ 
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The WO 99/60788 document ("Potts"), which is referred to in the fourth paragraph, is 
only cited by the amendment for tracking the image of a moving speaker - namely for a earner,, 
in motion not for a device that is motionless. 

Accordingly, Applicants have-** made, the admissions as characterized by the Examiner 

in the Final Office Action. 

Moreover, the Examiner misconstrues Applicants' claim*. The Examiner alleges that the 
claims recite -the camera & being motionless during operation" [emphasis added]. In lact. 
Claim I recites that the image pickup device b motionless during operation. Accordingly, the 
Examiner's statements about pan, tilt, xoom cameras are completely irrelevant to Applicants' 
claim- Since Ma.kin i. apparently cited only for this feature, Malkin, at least as applied, would 
also appear to be irrelevant to the claim. 

,n short. the Bomber wholly fails u, «»1 the elements olekbn 1 on the references; and 
makes fa.se and Mcvan, statements about Applicants' claims and Apples' amendment in 

support of the rejection. 

The Board is respectfully requested to remand this application to the Examiner .or 

issuance of a rejection that complies with 37 CFR 1 .104. 


Claim 5 

The off.« aetion makes no explicit rejection of this claim as distinguished ««n the other 

.^.p^ffli.- — «-» Thatisaoxwhatisrecited. What is recited is a system 
for electrically manipulating the image rip*. H is this manipulation that provides pan. tilt 
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and/or zoom functions . The image pickup is still the same motionless device that was cited in 
claim L 

Accordingly, as mentioned above, the references cited for pan, tilt, zoom cameras are 
irrelevant. The Examiner has I ailed to indicate any reference that teaches or suggests pan, tilt, 
zoom functions being generating in a motionless image pickup device. Applicants accordingly 
respectfully submit that the iixaminer has failed to meet his burden ol'making & prima facie case 
against ihis claim. 

Claim 2 

The Examiner fails to indicate where the references teach or suggest these features. 
Applicants accordingly respectfully submit that the Examiner has failed to meet his burden of 
making a prima facie case against this claim. 

Claim 3 

The Examiner fails to indicate where the references teach or suggest these features. 
Applicants accordingly respectfully submit that the Examiner ha.s I'tnled to meet his burden of 
making a prima facie case against this claim. 

Claim 4 

The Examiner fails to indicate where the references teach or suggest this feature or the 
resulting functional advantage. Applicants accordingly respectfully submit that the Examiner has 
failed to meet his burden ol'making a prima facie case against this claim. 
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Claim 6 

The Uxominer has failed to indicate any reference that teaches or suggests this feature. 
Applicants accordingly respectfully submit that the Examiner has Tailed to meet his bm-den of 
making a prima facie case against this claim. 

Claim 8 

The Examiner fails to indicate where the references teach or suggest the recited features 
or the resulting functional advantage. Applicants accordingly respectfully submit that the 
Examiner has failed to meet his burden of making a prima facie case against this claim. 

Claims 1 0 & 24 

The relined image signals are a Junctional advantage. Accordingly, the claim 
distinguishes even more clearly over the references than claim 1 does. 

The Examiner has completely failed to indicate where these recitations or the 
accompanying functional, advantage are taught or suggested in the references. Applicants 
accordingly respectfully submit that the Examiner has failed to meet his burden of making a 
prima facie case against this claim. 

Claim 24 is analogous. 
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Clairnli 

This claim depends from claim 1 0 and contains recitations analogous to those orclaims 2 
and 5. Accordingly the arguments applicable to those claims apply to this claim. 

Claim 12 

The Examiner fails to indicate where the references teach or suggest this feature or Lhe 
functional advantage of varying a field of view. Applicants accordingly respectfully submit thai 
the Examiner has failed to meet his burden of making iprima facie case against this claim. 

C laim 14 

This claim depends from claim 10, but contains recitations similar to those of claim 8. 
Arguments applicable Lo claim 8 are therefore applicable to claim J 4. 

Claim 15 

The Examiner fails to indicate where the references teach or suggest this feature. 
Applicants accordingly respectfully submit that the Examiner has failed u> meet his burden of 
making a prima facie case against this claim. 

Claim 16 

■ This claim includes, inter alia, the staiioiiary/moiioniess limitation discussed with respect 
claim 1 and the pan/lilt/zoom system-limitation discussed with respect to claim 5. The 
:jcction is accordingly deficient with respect to these limitations, as discussed before. 


to 


rei 


PAGE 16/28 • RCVD AT 2(1712006 5:29:32 PM (Eastern Standard Time] * SVR:USPTO€FXRF-6/37 • DNIS:2738300* CSID:1 914 332 7719* DURATION (mm-ss):04-38 


Friday, February 17, 2006 5:31 PM Anne Barschall 1-914-332-7719 p.17 


Claim 17 

The Kxamincr fails to indicate where the references teach or suggest this feature. 
Applicants accordingly respectfully submit that the Examiner has Tailed lo meet his burden of 
making a prima facie case against this claim. 

Claim 18 

The Examiner fails to indicate where the references teach or suggests this feature. 
Applicants accordingly respectfully submit thai the Examiner has failed to meet his burden of 
making a prima facie case against this claim. 

Claim 1 9 

The Examiner fails to indicate where the references teach or suggests this feature. 
Applicants accordingly respectfully submit that the Examiner has failed to meet his burden of 
making a prima facie case against this claim. 

Claim 20 

The Examiner fails to indicate where the references teach or suggests these features. 
Applicants accordingly respectf ully submit that the Examiner has faded to meet his burden of 
making a. prima facie case against this claim. The functional advantage of changing field of view 
has also not been read on the references in paragraph 8 or 9- 
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Claim 21 

The Examiner fails to indicaLe where the references teach or suggests these features. 
Applicants accordingly respectfully submit that the Examiner has failed to meet his burden of 
making a prima facie case against this claim. 


Claim 22 

The Examiner fails to indicate where the references teach or suggests this feature. 
Applicants accordingly respectfully submit that the Examiner has failed to meet his burden of 
making a prima facie case against this claim. 


Claim 23 

The Examiner fails to indicate where the references teach or suggests Ibis feature. 
Applicants accordingly respectfully submit that the Examiner has failed to meet his burden of 
making a prima facie case against this claim. 
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Vm. CONCLUSION 

Applicants respectfully submit that they have answered each issue raised by The lixaminer 
and that the application is accordingly in condition for allowance. Such allowance is therefore 
respectfully requested. 


Respectful 1 y subm itted , 



Anne H. Barschall 
Reg. No. 31,089 
(914) 332-1019 
fa* 914-332-7719 
February 17. 2006 
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CLAIM APPENDIX 


i I . (previously presented) A video conferencing system comprising: 

i a stationary image pickup device, remaining motionless during operation, for generating 

3 image signals representative of an image; 

i an audio pickup device for generating audio signals rcprcscniaii ve of sound from an audio 

s source; and 

6 means for processing said image signals and said audio signals to determine a direction 

7 of the audio source relative to a reference point, the determination of direction depending al least 
a at times on the image signals. 


1 2. (previously presented) The video conferencing system of claim J wherein said processing 

2 means comprises: 

3 an audio source localization system; 

4 a computer vision person detection system; and 
s a multimodal speaker detection system. 

1 3. (previously presented) The video conferencing system of claim 2, further comprising an 

2 integrated housing tor an integrated video conferencing system incorporating the image pickup 

3 device, the audio pickup device, and the processing means. 
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4. (original) The video conferencing system of claim 3, wherein the integrated housing is sized 
for being portable. 

1 5. (previously presented) The video conferencing system of claim 1, further comprising an 

2 electronic pan lilt zoom system for electronically manipulating the image signals to effectively 

3 provide at least one of variable pan, lilt, and zoom functions. 

1 6. (previously presented) The video conferencing system of claim 1, wherein the image pickup 

2 device is a stationary camera lhal remains motionless during operation of the video conferencing 
a system. 

7, (previously presented) The video conferencing .system of claim 1, wherein the processing 
means provides conlrol signals to an electronic pan tilt zoom system. 

1 8. (previously presented) The video conferencing system of claim 2, wherein the audio koutcc 

2 localization system detects the movement of the audio source when the audio source moves 

3 relative to the reference point, and, in response to the movement, the audio source localization 
a system causes a change in a Held of view of the image pickup device. 

9. (previously presented) The video conferencing system of claim 1, wherein the audio pickup 
device is comprised of an array of two microphones. 
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10 


1 1 0. (previously presented) A method comprising the steps of: 

2 generating, at a stationary image pickup device, remaining motionless during operation, 

3 image signals representative of an image; 

generating, at an audio pickup device, audio signals representative of sound from an 

5 audio source; 

processing tbe image signals and the audio signals to determine a direction or the audio 

7 source relative to a relcrence point, the determination oldirection depending at least at times on 

8 the image signals: 

manipulating the image signals 10 produce refined image signals depending on the 

determined direction; and 

outputting said relincd image signals. 


11. (previously presented) The method of claim 10 further comprising the .steps of: 

applying said audio signals to an audio souroo localization system; 

applying said image signals to a computer vision person detection system; 

processing said audio signals and said image signals with a multimodal speaker detection 
system to determine the direction of the audio source; 

generating control signals based on the determined direction of the audio source; 

applying the ccmtrol si&na1s to an electronic pan tilt zoom system to mimic the effect of at 
least one function of a movable camera, said function selected from the group consisting 
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10 


panning, tilling, and zooming said movable camera; and 

providing an output from said electronic pan tilt zoom system. 

12. (previously presented) The method of claim 10, wherein manipulating the image signals 
includes varying a field of view of the image pickup device in response to the control signals. 

1 3. (original) The method of claim 10, wherein processing the audio signals includes determining 
an audio based direction or the audio source based on the audio signals. 

, 14. (previously presented) The method of claim 10, wherein processing the audio signals 
2 include* detecting tbe movement of tb. audio source when the audio source moves; and 
manipulating the image signals includes causing electronically, in response to the 
4 movement, a variation in a field of view of the image pickup device. 

, 15. (previously presented) The method »f olafcn 13, wherein processing the image signals 

, includes generating control signals depending on the audio based direction, and manipulating the 

., image includes electronically panning, tilting, and/or /.ooming said image pickup device 

4 depending on the control signals. 

, 16. (previously presented) A video conferencing system comprising: 

tnicrophoncs for generating audio signals representative of sound from a speaker; 
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a stationary video camera, remaining motionless during operation, for generating video 
4 signals representative of a video image; 

an electronic pan tilt zoom system fiir manipulating video images to produce the visual 

o effects of panning, tilling, and/or zooming; 

a processor for processing the video signals and the audio signal., to determine a direction 
x of a speaker relative to a reference point and supplying control signals to the electronic pan tilt 
• zoom system for producing images that include the speaker in the field of view of the camera, the 
,o determination of direction depending at leasL at times on the video signals, tine control signals 
„ being generated based on the determined direction of the speaker; and 

a transmitter fin- transmitting atuho and video signals for video conferencing. 


, 17. (previously presented) The video conferencing system of claim 1, wherein at times the 

a determination of the direction of the audio source depends on both the image signals and the 

? audio signals. 

, 18. (previously presented) The video conferencing system of claim 1. wherein the processing 

2 includes determining the movement of the audio source depending at least at times on the image 

y signals. 

, 19. (previously presented) The video conferencing system of claim 1 , wherein the processing 

, ine.udes tracking the position of the audio source when the audio source moves, tho tracking 
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3 depending at least at times on the image signals. 

, 20. (previously presented) The video conferencing system of claim 2, wherein the computer 
2 vision person detection system detects the movement of the audio source when the audio source 
moves relative to the reference point, and, in response to the movement, the computer vision 

4 person detection system causes a change in a field .,f view of the image pickup device. 


21 


. (previously presented) The method of claim 1 0. wherein processing the image signals further 


includes: 

detecting the movement of the audio source when the audio source moves; and 
musing electronically, in response to the movement, an variation in a field of view of the 
image pickup device. 

22. (previously presented) The method of claim 10, wherein ihc processing includes determining 
the movement of the audio source depending at lea* at times on the image signals. 
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23. (previously presented) The method of claim 10, wherein the processing includes tracking the 
position of the audio source when the audio source moves, the tracking depending at least at 
times on the image signals. 

i 24. (previously presented) A video conferencing system, comprising: 

a stationary image pickup device, remaining motionless during operation, for generating 
3 image signals representative of an image; 

an audio pickup device for generating audio signals representative of sound from an audio 

5 source; 

6 mean* for processing the image signals and the audio signals Ui determine a direction of 

7 the audio source relaLive to a reference point, the determination of direction depending at least at 

8 times on the image signals; 

9 means for manipulating the image signals to produce refined image signals depending on 

10 the determined direction; and 

i, an output for outputting said refined image signals. 

25. (previously presented) The video conferencing system of claim 9, wherein the array of 
microphones includes only two microphones. 
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